Support yaml merge operator #1040
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Full logs: https://github.com/onthegomap/planetiler/actions/runs/11091650043 |
Add tests from the [YAML Merge Key Draft](https://yaml.org/type/merge.html)
zstadler
reviewed
Sep 29, 2024
| */ | ||
| private static void handleMergeOperator(Object parsed) { | ||
| if (parsed instanceof Map<?, ?> map) { | ||
| Object toMerge = map.remove("<<"); |
There was a problem hiding this comment.
Could this code, or similar, be used check if the merge operator is the first key in the mapping?!
Suggested change
| Object toMerge = map.remove("<<"); | |
| // Check if the first key is a merge operator using an iterator | |
| Iterator<String> keyIterator = map.keySet().iterator(); | |
| Object toMerge = null; | |
| if (keyIterator.next().equals("<<")) { | |
| toMerge = map.remove("<<"); | |
| } |
There was a problem hiding this comment.
Actually, when I read the merge operator draft again, it does not prohibit placing the merge operator at any location in a map. Neither does it prohibit multiple instances of merge operators in the same map.
Here is a YAML example where the use of the merge operator is not restricted, as in this suggestion:
- base: &base
nested_mapping: &base_nested_mapping
key1: value1
nested_sequence:
- item1
- item2
- extension: &extension
nested_mapping:
<<: *base_nested_mapping
key2: value2
nested_sequence:
- item3
- item4
- merged_using_array:
<<: [ *extension, *base]
- merged_separately:
<<: *extension
<<: *base
- merged_interleaved:
start: marker1
<<: *extension
mid: marker2
<<: *base
end: marker3According to YAML Lint, it would expand to
- base:
nested_mapping:
key1: value1
nested_sequence:
- item1
- item2
- extension:
nested_mapping:
key1: value1
nested_sequence:
- item3
- item4
key2: value2
- merged_using_array:
nested_mapping:
key1: value1
nested_sequence:
- item3
- item4
key2: value2
- merged_separately:
nested_mapping:
key1: value1
nested_sequence:
- item3
- item4
key2: value2
- merged_interleaved:
start: marker1
nested_mapping:
key1: value1
nested_sequence:
- item3
- item4
key2: value2
mid: marker2
end: marker3Please feel free to reject this suggestion
There was a problem hiding this comment.
Yeah I noticed that playing around with pyyaml. Since this approach happens post-parsing and the parser throws a clear exception in this case and you can easily switch to <<: [*a, *b] I think it's an OK limitation. A more fully-thought-out merge operator might support things like merging maps in the order they are specified when the operator appears more than once (and working for lists) but that would require support at the parser level.
|
It is worthwhile checking that the code can identify and avoid circular references such as include_when: &include
<< : *include
power:
- towerCircular references not specific to the merge operator. They can also occur with anchors and aliases: include_when: &include
- *include
- power:
- tower |
|
|
Thanks @msbarry for taking care of this request so quickly! BTW, is there some debug switch that can show what the expanded custom YAML file looks like? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support yaml merge operator so you can do things like this in custom yaml files to reuse a block and override values in it.
Fixes #1038